1. Field of the Invention
The invention relates generally to image processing. More specifically, the invention relates to encoding and quantization for image compression.
2. Description of the Related Art
When an image is captured of a scene, environment or object, the image is represented by an array of locations known as pixels. Each pixel of an image has a value relating to one or more color planes. When an image is captured by an imaging device such as a digital camera it is often captured into a Color Filter Array (CFA) pattern known as the Bayer pattern. In the Bayer pattern, each pixel location is an intensity value relating to only one of the three primary rendering colors (Red (R), Green (G) and Blue (B)). The Bayer pattern arranges pixels as follows:
G R G R . . . PA1 B G B G . . . PA1 G R G R . . . PA1 B G B G . . .
Since there are twice as many G related pixels as either of B or R pixels, the G or Green color plane may be considered as two separate color planes G1 (G pixels on the same row as R pixels) and G2 (G pixels on the same row as B pixels). Thus, a Bayer pattern "raw" image can be considered to contain four independent color planes. To obtain a full resolution color image (e.g., for rendering), each pixel location should have all three R, G and B components, not just one. To achieve this a process known as color interpolation is employed where missing color components for a pixel are estimated based on neighboring pixels.
After an image is captured and perhaps color interpolated then the image is often "compressed" or reduced in terms of the total number of bits that would be needed to store or transmit the image. Such image compression is ordinarily applied after color interpolation, but it would be advantageous in certain instances to perform suitable compression before color interpolation while an image is still in the Bayer pattern raw image format. Image compression plays a key role in multimedia applications such as video conferencing, digital imaging and video streaming over a network. Image compression schemes for such applications should be designed to reduce the bit-rate of storage or transmission of the image while maintaining acceptable image quality for the specific application.
Image compression techniques can be classified as either "lossy" or "lossless". With lossless compression, the original image prior to compression can be exactly recovered when the compressed image is decompressed. Consequently, lossless techniques, whose compression ratios depend upon the entropy of an image, do not ordinarily achieve high compression ratios and, since they preserve a high percentage of original image information, may also be computationally expensive. By contrast, lossy compression schemes provide only an approximation of the original image. Thus, with lossy compression, greater compression ratios can be achieved but often with loss in image quality compared to lossless techniques. One such lossy technique is a transform-based coding known as JPEG (Joint Photographic Experts Group) which transforms pixels of an input image using the well-known Discrete Cosine Transform (DCT). The resulting transformed pixel values are quantized or mapped to smaller set of values in order to achieve compression. The quality of a compressed image that is decompressed will depend greatly on how the quantization of the transformed pixels are performed. The compression ratio (the size of the original raw image compared to the compressed image) will also be affected by the quantization, but can be enhanced by the binary encoding of the data after quantization.
Further, high compression ratio algorithms such as JPEG suffer from deficiencies such as "blocking artifacts". For these algorithms, an image is divided into blocks of pixels such as 8.times.8, or 16.times.16 blocks. These blocks are processed independently of each other and thus, between blocks, there is an observable discontinuity in luminance or color which constitutes a "blocking artifact".
These and other image compression schemes which achieve high compression ratios and sometimes, also acceptable decompressed image quality, operate on images better when the images are in "luminance-chrominance" format. Unlike Bayer pattern or color interpolated full RGB image "spaces" (i.e. formats) which represent a pixel's color as a selected mixture of primary colors (such as red, green and blue), luminance-chrominance format images define each pixel in terms of hue and saturation levels. Since imaging devices such as digital cameras ordinarily capture images in Bayer pattern format, an image must first be color interpolated into full resolution [RGB] and then have its "color space" converted into a luminance-chrominance format such as YCrCb before luminance-chrominance techniques can be applied. Such color interpolation and color space conversion is often cost-prohibitve as well as time-consuming and thus, not desirable.
FIG. 1 shows one such traditional approach. An original image 100, captured for instance from a device such as a digital camera, is ordinarily in a raw image format such as the Bayer pattern. As such, each pixel will not have full color representation. Thus, the image is passed, either entirely at once or, block by block, to a color interpolation pixel procedure 110. The color interpolation procedure generates full-color pixels from the image 100, each pixel of which has full color resolution (e.g., R, G, and B components). The full color image is then color space converted (block 120) from RGB to YUV or other appropriate space. Such conversion may improve the compression ratio that is achievable. Once converted, the image is then passed to a primary compression procedure (block 130). Such compression may include a variety of procedures, such as JPEG or Fourier analysis, etc., but often has, as a component, a procedure known as quantization. An image is quantized by mapping a range of values representing the image pixels to a smaller range of values. After compression, the compressed image values can be encoded (block 140) such that they are suitable for transmission or storage.
This traditional approach suffers several drawbacks. First, the entire procedure is computationally complex particularly in the color interpolation and color space conversion. The color space conversion alone requires (for RGB to YCrCb space, for example) nine multiplications and six additions for each and every pixel. Often, such complicated techniques are unable to be effectively implemented in small, cost-conscious devices such as digital cameras.
If images are to be compressed on a digital camera or other imaging device, the compression techniques described above would be ineffective or prohibitive. Thus, there is a need for an image quantization and compression technique which is computationally inexpensive so as to reduce the cost of the digital cameras on which they are employed. To avoid the need for first performing color interpolation, a quantization and compression method should be developed that can be applied directly to Bayer pattern raw image data that is generated by portable imaging devices and that can exploit the correlation between different color planes to achieve high compression ratios. Additionally, there is a need for enhancing the speed of quantization and compression so that the image capture and store in local memory or transfer from the imaging device can be performed more in real-time while still preserving image quality.